Multifunctional integrated image sensor and application to virtual interface technology

ABSTRACT

An electronic camera including an electronic imaging sensor providing an output representing an imaged field, a first imaging functionality employing the electronic imaging sensor for taking a picture of a scene in the imaged field, a second imaging functionality employing the electronic imaging sensor for data entry responsive to user hand activity and a user-operated imaging functionality selection switch operative to enable a user to select operation in one of the first and second imaging functionalities.

REFERENCE TO CO-PENDING APPLICATIONS

[0001] This application claims priority from the following co-pendingU.S. patent applications:

[0002] U.S. Provisional application Ser. No. 60/392,376, entitled“Virtual Interface Implementation and Application”, filed Jun. 26, 2002and U.S. Provisional application Ser. No. 60/438,327, entitled“Multifunctional Integrated Image Sensor and Application to VirtualInterface Technology”, filed Jan. 7, 2003.

BACKGROUND OF THE INVENTION

[0003] The following patents and publications are believed to representthe current state of the art:

[0004] U.S. Pat. Nos. 6,377,238; 6,281,238; 5,767,842; 5,457,550;5,181,181 and 4,843,568.

[0005] Foreign Patent Documents: WO 00/21024; EP 0982 676 A1; DE 298 02435 U1; WO 01/93182 and WO 02/054169.

SUMMARY OF THE INVENTION

[0006] The present invention relates to data input methods and apparatusgenerally.

[0007] There is thus provided in accordance with a preferred embodimentof the present invention an electronic camera including an electronicimaging sensor providing an output representing an imaged field, a firstimaging functionality employing the electronic imaging sensor for takinga picture of a scene in the imaged field, a second imaging functionalityemploying the electronic imaging sensor for data entry responsive touser hand activity and a user-operated imaging functionality selectionswitch operative to enable a user to select operation in one of thefirst and second imaging functionalities.

[0008] There is also provided in accordance with another preferredembodiment of the present invention a portable telephone includingtelephone functionality, an electronic imaging sensor providing anoutput representing an imaged field, a first imaging functionalityemploying the electronic imaging sensor for taking a picture of a scenein the imaged field, a second imaging functionality employing theelectronic imaging sensor for data entry responsive to user handactivity and a user-operated imaging functionality selection switchoperative to enable a user to select operation in one of the first andsecond imaging functionalities.

[0009] There is further provided in accordance with yet anotherpreferred embodiment of the present invention a digital personalassistant including at least one personal digital assistantfunctionality, an electronic imaging sensor providing an outputrepresenting an imaged field, a first imaging functionality employingthe electronic imaging sensor for taking a picture of a scene in theimaged field, a second imaging functionality employing the electronicimaging sensor for data entry responsive to user hand activity and auser-operated imaging functionality selection switch operative to enablea user to select operation in one of the first and second imagingfunctionalities.

[0010] In accordance with another preferred embodiment of the presentinvention the electronic imaging sensor includes a CMOS sensor.

[0011] Preferably, the first imaging functionality incorporates a firstlens arrangement upstream of the electronic imaging sensor.Additionally, the second imaging functionality incorporates a secondlens arrangement upstream of the electronic imaging sensor.

[0012] In accordance with yet another preferred embodiment of thepresent invention the second imaging functionality incorporates aninfrared light generator for defining a light beam, impingement of whichresulting from the user hand activity produces a light patternrepresentative of the data. Additionally, the second imagingfunctionality also incorporates at least one infrared passing lightfilter associated with the electronic imaging sensor.

[0013] In accordance with still another preferred embodiment of thepresent invention the second imaging functionality incorporates aprocessor receiving an output from the electronic imaging sensor forproviding a data entry output. Additionally, the processor isincorporated together with the electronic imaging sensor on a singlechip.

[0014] In accordance with another preferred embodiment of the presentinvention the first functionality is a color photography functionality.Additionally or alternatively, the second functionality is an infraredbeam impingement sensing data entry functionality.

[0015] In accordance with yet another preferred embodiment of thepresent the user-operated imaging functionality selection switch is amechanically responsive switch which selectably mechanically associatesthe first and second lens arrangements upstream of the electronicimaging sensor. Alternatively or additionally, the user-operated imagingfunctionality selection switch is a mechanically responsive switch whichselectably mechanically associates selected filters with the electronicimaging sensor.

[0016] In accordance with still another preferred embodiment of thepresent invention the user-operated imaging functionality selectionswitch is an electronically responsive switch which selectablyelectronically determines responsiveness to selected outputs frompredetermined portions of the electronic imaging sensor.

[0017] In accordance with another preferred embodiment of the presentinvention the electronic camera also includes an array of red, green,blue and infrared passing pixelwise elements arranged over theelectronic imaging sensor. Additionally or alternatively, the electroniccamera also includes a data entry template projector employed by thesecond imaging functionality. Preferably, the electronic camera alsoincludes an illuminator employed by the second imaging functionality.Additionally or alternatively, the electronic camera also includes asynchronized illumination power variation functionality.

[0018] In accordance with another preferred embodiment of the presentinvention the portable telephone also includes an array of red, green,blue and infrared passing pixelwise elements arranged over theelectronic imaging sensor. Additionally or alternatively, the portabletelephone also includes a data entry template projector employed by thesecond imaging functionality. Preferably, the portable telephone alsoincludes an illuminator employed by the second imaging functionality.Additionally or alternatively, the portable telephone also includes asynchronized illumination power variation functionality.

[0019] In accordance with another preferred embodiment of the presentinvention the digital personal assistant also includes an array of red,green, blue and infrared passing pixelwise elements arranged over theelectronic imaging sensor. Additionally or alternatively, the digitalpersonal assistant also includes a data entry template projectoremployed by the second imaging functionality. Preferably, the digitalpersonal assistant also includes an illuminator employed by the secondimaging functionality. Additionally or alternatively, the digitalpersonal assistant also includes a synchronized illumination powervariation functionality.

[0020] There is also provided in accordance with another preferredembodiment of the present invention an electronic camera including anelectronic imaging sensor providing an output representing an imagedfield and an array of red, green, blue and infrared passing pixelwiseelements arranged over the electronic imaging sensor.

[0021] There is further provided in accordance with still anotherpreferred embodiment of the present invention a portable telephoneincluding telephone functionality, an electronic imaging sensorproviding an output representing an imaged field and an array of red,green, blue and infrared passing pixelwise elements arranged over theelectronic imaging sensor.

[0022] There is also provided in accordance with another preferredembodiment of the present invention a digital personal assistantincluding at least one personal digital assistant functionality, anelectronic imaging sensor providing an output representing an imagedfield and an array of red, green, blue and infrared passing pixelwiseelements arranged over the electronic imaging sensor.

[0023] There is even further provided in accordance with yet anotherpreferred embodiment of the present invention an electronic cameraincluding an electronic imaging sensor providing an output representingan imaged field, a first imaging functionality employing the electronicimaging sensor in a visible radiation imaging mode for taking a pictureof a scene in the imaged field and a second imaging functionalityemploying the electronic imaging sensor in an infrared radiation sensingmode for data entry responsive to user hand activity.

[0024] There is also provided in accordance with another preferredembodiment of the present invention a portable telephone includingtelephone functionality, an electronic imaging sensor providing anoutput representing an imaged field, a first imaging functionalityemploying the electronic imaging sensor in a visible radiation imagingmode for taking a picture of a scene in the imaged field and a secondimaging functionality employing the electronic imaging sensor in aninfrared radiation sensing mode for data entry responsive to user handactivity.

[0025] There is further provided in accordance with yet anotherpreferred embodiment of the present invention a digital personalassistant including at least one personal digital assistantfunctionality, an electronic imaging sensor providing an outputrepresenting an imaged field, a first imaging functionality employingthe electronic imaging sensor in a visible radiation imaging mode fortaking a picture of a scene in the imaged field and a second imagingfunctionality employing the electronic imaging sensor in an infraredradiation sensing mode for data entry responsive to user hand activity.

[0026] In accordance with another preferred embodiment of the presentinvention the electronic camera includes an array of red, green, blueand infrared passing pixelwise elements arranged over the electronicimaging sensor.

[0027] In accordance with yet another preferred embodiment of thepresent invention the portable telephone includes an array of red,green, blue and infrared passing pixelwise elements arranged over theelectronic imaging sensor.

[0028] In accordance with yet another preferred embodiment of thepresent invention the digital personal assistant includes an array ofred, green, blue and infrared passing pixelwise elements arranged overthe electronic imaging sensor.

[0029] Preferably, in the first imaging functionality output of thepixels underlying the red, green and blue passing elements are employed.Additionally, output of the pixels underlying the infrared passingelements is employed to prevent distortion of color response of theimaging sensor. Additionally, in the second imaging functionality outputof the pixels underlying the infrared passing elements are employed.

[0030] In accordance with yet another preferred embodiment of thepresent invention the electronic camera also includes a processor. Inaccordance with still another preferred embodiment of the presentinvention the electronic camera forms part of a telephone.Alternatively, the electronic camera forms part of a personal digitalassistant. Alternatively, the electronic camera forms part of awristwatch.

[0031] In accordance with another preferred embodiment of the presentinvention the portable telephone also includes a processor.

[0032] In accordance with yet another preferred embodiment of thepresent invention the digital personal assistant also includes aprocessor.

[0033] In accordance with still another preferred embodiment of thepresent invention the processor includes the following functionality: aseach pixel value is acquired, determining, using pixel coordinates,whether that pixel lies within a predefined keystroke region, acquiringpixel values for the pixel coordinates lying within the predefinedkeystroke region, adding or subtracting each of the pixel values to orfrom a pixel total maintained for each the keystroke region based ondetermining a pixel function of each pixel, comparing the pixel totalfor each the keystroke region with a current key actuation threshold, ifthe pixel total exceeds the key actuation threshold for a givenkeystroke region in a given frame and in the previous frame the pixeltotal did not exceed the key actuation threshold for that keystrokeregion, providing a key actuation output and if the pixel total does notexceed the key actuation threshold for a given keystroke region in agiven frame and in the previous frame the pixel total did exceed the keyactuation threshold for that keystroke region, providing a keydeactuation output.

[0034] In accordance with another preferred embodiment of the presentinvention the determining whether that pixel lies within a predefinedkeystroke region is made by employing a pixel index table whichindicates for each pixel, whether that pixel lies within a predeterminedkeystroke region and, if so, within which keystroke region it lies.Alternatively, both of the determining steps employ the pixel indextable.

[0035] In accordance with yet another preferred embodiment of thepresent invention the pixel total is maintained for each keystrokeregion in a keystroke region accumulator table.

[0036] In accordance with still another preferred embodiment of thepresent invention the comparing employs a keystroke region thresholdtable.

[0037] In accordance with another preferred embodiment of the presentinvention the processor also includes the following functionality: onceall of the pixels in a frame have been processed, determining an updatedbackground level for a frame and determining a key actuation thresholdfor the keystroke region threshold table by subtracting the updatedbackground level from a predetermined threshold level which isestablished for each keystroke region. Additionally, the pixel functionincludes adding the pixel values of a plurality of pixels in thekeystroke region.

[0038] In accordance with yet another preferred embodiment of thepresent invention the processor is operative to determine the “center ofgravity” of pixel values of pixels in the two-dimensional image sensor.Preferably, the processor includes the following functionality: as eachpixel value is acquired, determining, using the pixel coordinates,whether that pixel lies within a predefined active region, acquiringpixel values for various pixel coordinates and determining the “centerof gravity” of the pixel values.

[0039] In accordance with still another preferred embodiment of thepresent invention determining the “center of gravity” is achieved bymultiplying the pixel values by X and Y values representing thegeographic position of each pixel, summing the results along mutuallyperpendicular axes X and Y, summing the total of the pixel values forall relevant pixels for the active region and dividing the summedresults by the total of the pixel values to determine the X and Ycoordinates of the “center of gravity”, which represents a desiredengagement location.

[0040] Preferably, the pixel values are thresholded prior to summingthereof.

[0041] There is yet further provided in accordance with anotherpreferred embodiment of the present invention a wristwatch includingwristwatch functionality, an electronic imaging sensor providing anoutput representing an imaged field, a first imaging functionalityemploying the electronic imaging sensor for taking a picture of a scenein the imaged field, a second imaging functionality employing theelectronic imaging sensor for data entry responsive to user handactivity and a user-operated imaging functionality selection switchoperative to enable a user to select operation in one of the first andsecond imaging functionalities.

[0042] There is also provided in accordance with another preferredembodiment of the present invention a vehicle including a frame, a drivetrain and a chassis, including at least one lockable entry portal andportal unlocking functionality and an access control assembly forgoverning authorized access to the at least one lockable entry portalincluding at least one at least partially light transmissive user fingerengagement surface accessible from outside the chassis, an illuminatoroperative to illuminate at least one user finger engagement planeadjacent the at least one user finger engagement surface, atwo-dimensional imaging sensor viewing the at least one user fingerengagement surface, from a location inwardly of the at least one fingerengagement surface, for sensing light from the illuminator scattered byengagement of a user's finger with the at least one finger engagementsurface, and a data entry processor receiving an output from thetwo-dimensional imaging sensor and providing a data entry input to theportal unlocking functionality.

[0043] There is further provided in accordance with yet anotherpreferred embodiment of the present invention access control apparatusfor use with a vehicle including a chassis having at least one lockableentry portal and portal unlocking functionality and also including atleast one at least partially light transmissive user finger engagementsurface accessible from outside the chassis, the access controlapparatus including an illuminator operative to illuminate the at leastone user finger engagement surface, a two-dimensional imaging sensorviewing the at least one user finger engagement surface from a locationinwardly of the at least one finger engagement surface for sensing lightfrom the illuminator scattered by engagement of a user's finger with theat least one finger engagement surface and a data entry processorreceiving an output from the two-dimensional imaging sensor andproviding a data entry input to the portal unlocking functionality.

[0044] There is further provided in accordance with yet anotherpreferred embodiment of the present invention access control apparatusfor use with an enclosure having at least one lockable entry portal andportal unlocking functionality and also including at least one at leastpartially light transmissive user finger engagement surface accessiblefrom outside the enclosure, the access control apparatus including anilluminator operative to illuminate the at least one user fingerengagement surface, a two-dimensional imaging sensor viewing the atleast one user finger engagement surface from a location inwardly of theat least one finger engagement surface for sensing light from theilluminator scattered by engagement of a user's finger with the at leastone finger engagement surface and a data entry processor receiving anoutput from the two-dimensional imaging sensor and providing a dataentry input to the portal unlocking functionality.

[0045] There is yet further provided in accordance with still anotherpreferred embodiment of the present invention data entry apparatus foruse with at least one at least partially light transmissive user fingerengagement surface, the data entry apparatus including an illuminatoroperative to illuminate the at least one user finger engagement surface,a two-dimensional imaging sensor viewing the at least one user fingerengagement surface from a location inwardly of the at least one fingerengagement surface for sensing light from the illuminator scattered byengagement of a user's finger with the at least one finger engagementsurface and a data entry processor receiving an output from thetwo-dimensional imaging sensor and providing a data entry input.

[0046] Preferably, the illuminator is located at a location inwardly ofthe at least one engagement surface. Additionally or alternatively, theat least one finger engagement surface includes a one-dimensional arrayof a plurality of finger engagement surfaces. Alternatively, the atleast one finger engagement surface includes a two-dimensional array ofa plurality of finger engagement surfaces.

[0047] In accordance with another preferred embodiment of the presentinvention the vehicle also includes an illumination director cooperatingwith the illuminator for providing an illumination beam generallyparallel to the at least one finger engagement surface.

[0048] In accordance with yet another preferred embodiment of thepresent invention the access control apparatus also includes anillumination director cooperating with the illuminator for providing anillumination beam generally parallel to the at least one fingerengagement surface.

[0049] In accordance with still another preferred embodiment of thepresent invention the data entry apparatus also includes an illuminationdirector cooperating with the illuminator for providing an illuminationbeam generally parallel to the at least one finger engagement surface.

[0050] Preferably, the illumination director includes a prism.

[0051] There is also provided in accordance with another preferredembodiment of the present invention data entry apparatus for use with atleast one engagement surface, the data entry apparatus including a lightemitting data entry engagement element, which emits light only when itis in at least predetermined propinquity to the at least one engagementsurface, a two-dimensional imaging sensor viewing the at least oneengagement surface for sensing light emitted by the light emitting dataentry engagement element and a data entry processor receiving an outputfrom the two-dimensional imaging sensor and providing a data entryinput.

[0052] Preferably, the light emitting data entry engagement elementincludes an elongated element having an LED at an end thereof.Additionally, the LED includes an IR emitting LED. Additionally oralternatively, the data entry apparatus also includes a proximityswitch.

[0053] There is further provided in accordance with yet anotherpreferred embodiment of the present invention data entry apparatus foruse with at least one engagement surface, the data entry apparatusincluding a projector operative to illuminate the at least oneengagement surface and to define thereon at least one touchpad regionand at least one keyboard region, the touchpad region being defined by azero'th order diffracted image, an illuminator operative to illuminatethe at least one engagement surface, a two-dimensional imaging sensorviewing the at least one engagement surface from a location inwardly ofthe at least one engagement surface for sensing light from theilluminator scattered by engagement of a user's finger with the at leastone engagement surface and a data entry processor receiving an outputfrom the two-dimensional imaging sensor and providing a data entryinput.

[0054] Preferably, the processor includes the following functionality:as each pixel value is acquired, determining, using pixel coordinates,whether that pixel lies within a predefined keystroke region, acquiringpixel values for the pixel coordinates lying within the predefinedkeystroke region, adding or subtracting each of the pixel values to orfrom a pixel total maintained for each the keystroke region based ondetermining a pixel function of each pixel, comparing the pixel totalfor each the keystroke region with a current key actuation threshold, ifthe pixel total exceeds the key actuation threshold for a givenkeystroke region in a given frame and in the previous frame the pixeltotal did not exceed the key actuation threshold for that keystrokeregion, providing a key actuation output and if the pixel total does notexceed the key actuation threshold for a given keystroke region in agiven frame and in the previous frame the pixel total did exceed the keyactuation threshold for that keystroke region, providing a keydeactuation output.

[0055] There is even further provided in accordance with still anotherpreferred embodiment of the present invention a data input deviceincluding an illuminator operative to illuminate at least one engagementplane by directing light along the at least one engagement plane, atwo-dimensional imaging sensor, including an optical wedge element,viewing the at least one engagement plane from a location outside the atleast one engagement plane for sensing light from the illuminatorscattered by engagement of a data entry object with the at least oneengagement plane and a data entry processor receiving an output from thetwo-dimensional imaging sensor and providing a data entry input toutilization circuitry.

[0056] Preferably, the optical wedge element enables the imaging sensorto efficiently sense light from the engagement plane.

[0057] There is still further provided in accordance with yet anotherpreferred embodiment of the present invention a data input deviceincluding an illuminator operative to illuminate at least one engagementplane by directing light along the at least one engagement plane, atwo-dimensional imaging sensor, including a pair of oppositely orientedoptical wedge elements, viewing the at least one engagement plane from alocation outside the at least one engagement plane for sensing lightfrom the illuminator scattered by engagement of a data entry object withthe at least one engagement plane and a data entry processor receivingan output from the two-dimensional imaging sensor and providing a dataentry input to utilization circuitry.

[0058] There is also provided in accordance with still another preferredembodiment of the present invention a data input device including anilluminator operative to illuminate at least one engagement plane bydirecting light along the at least one engagement plane and toilluminate a proper positioning confirmation location on an engagementsurface, a two-dimensional imaging sensor, including, viewing the atleast one engagement plane from a location outside the at least oneengagement plane for sensing light from the illuminator scattered byengagement of a data entry object with the at least one engagement planeand a data entry processor receiving an output from the two-dimensionalimaging sensor and providing a data entry input to utilizationcircuitry.

[0059] Preferably, the imaging sensor senses the presence or absence ofillumination at the proper positioning confirmation location.Additionally, the data entry apparatus is operative to unpower itselfwhen the imaging sensor senses the absence of illumination at the properpositioning confirmation location.

[0060] There is further provided in accordance with yet anotherpreferred embodiment of the present invention gaming apparatus includingat least one game piece, a game template projector, operative to projecta game template onto a surface, an illuminator operative to illuminatethe surface and an imaging sensor viewing the surface for sensing lightfrom the illuminator reflected by at least one optically encodedreflector mounted on the at least one game piece.

[0061] In accordance with another preferred embodiment of the presentinvention each of the at least one reflector identifies one of the atleast one game piece and is operative to indicate a location of the atleast one game piece on the game template. Additionally oralternatively, the at least one reflector also identifies a spatialorientation of the game piece.

BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The present invention will be understood and appreciated morefully from the following detailed description, taken in conjunction withthe drawings in which:

[0063]FIGS. 1A & 1B are simplified pictorial illustrations of anelectronic camera constructed and operative in accordance with apreferred embodiment of the present invention in two different modes ofoperation;

[0064]FIGS. 2A & 2B are simplified pictorial illustrations of acamera-equipped cellular telephone, constructed and operative inaccordance with a preferred embodiment of the present invention in twodifferent modes of operation;

[0065]FIGS. 3A & 3B are simplified pictorial illustrations of acamera-equipped personal digital assistant, constructed and operative inaccordance with a preferred embodiment of the present invention in twodifferent modes of operation;

[0066]FIGS. 4A & 4B are simplified diagrammatic illustrations of theoperation of imaging functionality selection in accordance with apreferred embodiment of the present invention in two different modes ofoperation;

[0067]FIGS. 5A & 5B are simplified diagrammatic illustrations of theoperation of imaging functionality selection in accordance with anotherpreferred embodiment of the present invention in two different modes ofoperation;

[0068]FIG. 6 is a simplified diagrammatic illustration of a data entryobject engagement location sensing subsystem useful in the embodimentsof any of FIGS. 1A-5B;

[0069]FIG. 7 is a simplified flow chart illustrating operation of thedata entry object engagement location sensing subsystem of FIG. 6 inaccordance with a preferred embodiment of the present invention;

[0070]FIG. 8 is a simplified illustration of a preferred data structureemployed in the operation of the data entry object engagement locationsensing subsystem shown in FIG. 7;

[0071]FIG. 9 is a simplified pictorial illustration of outlines oftypical keystroke regions as sensed by a two-dimensional image sensorviewing a keyboard, such as the keyboard seen in FIG. 6;

[0072]FIG. 10 is a simplified pictorial illustration of outlines oftypical footprints of a typical light pattern occasioned by data entryobject engagement with several keystroke regions, such as those shown inFIG. 9;

[0073]FIGS. 11A, 11B and 11C are simplified illustrations of threealternative methodologies for determining the function of the pixelwithin the keystroke region in which it lies as shown in FIG. 7;

[0074]FIGS. 12A, 12B and 12C are simplified illustrations of traceswhich are useful in understanding FIGS. 11A, 11B and 11C;

[0075]FIG. 13 is a simplified flow chart illustrating operation of adata entry object engagement location sensing subsystem employed in atracking system and methodology constructed and operative in accordancewith a preferred embodiment of the present invention;

[0076]FIG. 14 is a simplified illustration showing synchronizedillumination power variation functionality useful in accordance with apreferred embodiment of the present invention;

[0077]FIG. 15 is a simplified partially pictorial, partiallydiagrammatic illustration of data entry apparatus embodied in a vehicleaccess control system;

[0078]FIG. 16 is a simplified partially pictorial, partiallydiagrammatic illustration of a variation in the data entry apparatus ofFIG. 15;

[0079]FIG. 17 is a simplified pictorial illustration of data entryapparatus constructed and operative in accordance with another preferredembodiment of the present invention;

[0080]FIGS. 18A and 18B are respectively a simplified pictorial and asimplified sectional illustration of data entry apparatus constructedand operative in accordance with another preferred embodiment of thepresent invention;

[0081]FIGS. 19A and 19B are respectively a simplified pictorial and asimplified sectional illustration of data entry apparatus constructedand operative in accordance with yet another preferred embodiment of thepresent invention;

[0082]FIGS. 20A and 20B are respectively a simplified pictorial and asimplified sectional illustration of data entry apparatus constructedand operative in accordance with still another preferred embodiment ofthe present invention;

[0083]FIG. 21 is a simplified pictorial illustration of data entryapparatus constructed and operative in accordance with a furtherpreferred embodiment of the present invention;

[0084]FIG. 22 is a simplified pictorial illustration of data entryapparatus constructed and operative in accordance with another preferredembodiment of the present invention;

[0085]FIG. 23 is a simplified pictorial illustration of gaming apparatusconstructed and operative in accordance with a preferred embodiment ofthe present invention; and

[0086]FIG. 24 is a simplified pictorial illustration of gaming apparatusconstructed and operative in accordance with another preferredembodiment of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0087] Reference is now made to FIGS. 1A & 1B, which are simplifiedpictorial illustrations of an electronic camera constructed andoperative in accordance with a preferred embodiment of the presentinvention in two different modes of operation. As seen in FIGS. 1A & 1B,an electronic camera 100, which may be of conventional constructionother than as specifically described hereinbelow, such as a NikonCOOLPIX 5700, available from Nikon Corporation, Tokyo, Japan, isequipped with an electronic imaging sensor module 102, which may includea CMOS array, for example, CMOS array OVT6130, available from OmnivisionTechnologies, Inc., 1341 Orleans Drive, Sunnyvale, Calif., USA,providing an output representing an imaged field.

[0088] A user-operated imaging functionality selection switch 104 ispreferably provided to enable a user to select operation in one of twopossible imaging functionalities. In a first imaging functionality,illustrated in FIG. 1A, the electronic imaging sensor module 102 isemployed to take a picture of a scene in the imaged field. In a secondimaging functionality, illustrated in FIG. 1B, the electronic imagingsensor module 102 is employed for data entry responsive to user handactivity.

[0089] Various embodiments of the first and second imagingfunctionalities will be described hereinbelow, with reference to FIGS.4A-5B. Preferably, but not necessarily, the second imagingfunctionality, as shown in FIG. 1B, employs a data entry templateprojector 106, which projects a data entry template, such as thatdesignated by reference numeral 108, onto a surface, such as a tablesurface 110. An illuminator 111 preferably is provided and is operativeto illuminate a region adjacent the data entry template 108. The imagingsensor module 102 views the data entry template region for sensing lightfrom the illuminator 111. The second imaging functionality alsopreferably incorporates processing functionality, an example of which isdescribed in applicant's Published PCT Application WO 02/054169 A2, thedisclosure of which is hereby incorporated by reference.

[0090] It is seen in FIG. 1A that when the electronic camera 100 isoperative in the first imaging functionality, also termed a picturemode, a digital picture 112 is provided. Similarly, it is seen in FIG.1B that when the electronic camera 100 is operative in the secondimaging functionality, also termed a data mode, a stream of data 114 isprovided.

[0091] Reference is now made to FIGS. 2A & 2B, which are simplifiedpictorial illustrations of a camera-equipped cellular telephone,constructed and operative in accordance with a preferred embodiment ofthe present invention in two different modes of operation. As seen inFIGS. 2A & 2B, a camera-equipped cellular telephone 200, which may be ofconventional construction other than as specifically describedhereinbelow, such as a Samsung SPH-1330, available from SamsungElectronics, Seoul, Korea, is equipped with an electronic imaging sensormodule 202, which may include a CMOS array, for example, CMOS arrayOVT6130, available from Omnivision Technologies, Inc., 1341 OrleansDrive, Sunnyvale, Calif., USA, providing an output representing animaged field.

[0092] A user-operated imaging functionality selection switch 204 ispreferably provided to enable a user to select operation in one of twopossible imaging functionalities. In a first imaging functionality,illustrated in FIG. 2A, the electronic imaging sensor module 202 isemployed to take a picture of a scene in the imaged field. In a secondimaging functionality, illustrated in FIG. 2B, the electronic imagingsensor module 202 is employed for data entry responsive to user handactivity.

[0093] Various embodiments of the first and second imagingfunctionalities will be described hereinbelow, with reference to FIGS.4A-5B. Preferably, but not necessarily, the second imagingfunctionality, as shown in FIG. 2B, employs a data entry templateprojector 206, which projects a data entry template, such as thatdesignated by reference numeral 208, onto a surface, such as a tablesurface 210. An illuminator 211 preferably is provided and is operativeto illuminate a region adjacent the data entry template 208. The imagingsensor module 202 views the data entry template region for sensing lightfrom the illuminator 211. The second imaging functionality alsopreferably incorporates processing functionality an example of which isdescribed in applicant's Published PCT Application WO 02/054169 A2, thedisclosure of which is hereby incorporated by reference.

[0094] It is seen in FIG. 2A that when the camera-equipped cellulartelephone 200 is operative in the first imaging functionality, alsotermed a picture mode, a digital picture 212 is provided. Similarly, itis seen in FIG. 2B that when the camera-equipped cellular telephone 200is operative in the second imaging functionality, also termed a datamode, a stream of data 214 is provided.

[0095] Reference is now made to FIGS. 3A & 3B, which are simplifiedpictorial illustrations of a camera-equipped personal digital assistant,constructed and operative in accordance with a preferred embodiment ofthe present invention in two different modes of operation. As seen inFIGS. 3A & 3B, a camera-equipped personal digital assistant 300, whichmay be of conventional construction other than as specifically describedhereinbelow, such as a Samsung SPH-1700, available from SamsungElectronics, Seoul, Korea, is equipped with an electronic imaging sensormodule 302, which may include a CMOS array, for example, CMOS arrayOVT6130, available from Omnivision Technologies, Inc., 1341 OrleansDrive, Sunnyvale, Calif., USA, providing an output representing animaged field.

[0096] A user-operated imaging functionality selection switch 304 ispreferably provided to enable a user to select operation in one of twopossible imaging functionalities. In a first imaging functionality,illustrated in FIG. 3A, the electronic imaging sensor module 302 isemployed to take a picture of a scene in the imaged field. In a secondimaging functionality, illustrated in FIG. 3B, the electronic imagingsensor module 302 is employed for data entry responsive to user handactivity.

[0097] Various embodiments of the first and second imagingfunctionalities will be described hereinbelow, with reference to FIGS.4A-5B. Preferably, but not necessarily, the second imagingfunctionality, as shown in FIG. 3B, employs a data entry templateprojector 306, which projects a data entry template, such as thatdesignated by reference numeral 308, onto a surface, such as a tablesurface 310. An illuminator 311 preferably is provided and is operativeto illuminate a region adjacent the data entry template 308. The imagingsensor module 302 views the data entry template region for sensing lightfrom the illuminator 311. The second imaging functionality alsopreferably incorporates processing functionality an example of which isdescribed in applicant's Published PCT Application WO 02/054169 A2, thedisclosure of which is hereby incorporated by reference.

[0098] It is seen in FIG. 3A that when the electronic camera 300 isoperative in the first imaging functionality, also termed a picturemode, a digital picture 312 is provided. Similarly, it is seen in FIG.3B that when the electronic camera 300 is operative in the secondimaging functionality, also termed a data mode, a stream of data 314 isprovided.

[0099] Reference is now made to FIGS. 4A & 4B, which are simplifieddiagrammatic illustrations of the operation of imaging functionalityselection in accordance with a preferred embodiment of the presentinvention in two different modes of operation. As seen in FIGS. 4A & 4B,one implementation of the invention involves the use of a first andsecond selectably positionable optical assemblies which may beselectably placed in front of an electronic imaging sensor module 402.

[0100] In the illustrated embodiment, the sensor module 402 comprises atwo-dimensional CMOS detector array 404 which is overlaid by an array ofcolor filters 406, which pass infrared radiation. A lens 408 focuseslight from an imaging field onto array 404.

[0101]FIG. 4A shows a first selectably positionable optical assembly410, preferably comprising an IR blocking filter 412, aligned with lens408. The imaging functionality of FIG. 4A is suitable for picturetaking, since it prevents IR radiation from distorting the colorresponse of the sensor module 402.

[0102]FIG. 4B shows a second selectably positionable optical assembly420, preferably comprising a visible radiation blocking filter 422, anIR band pass filter 424 and a field lens 426, aligned with lens 408. Theimaging functionality of FIG. 4B is suitable for data input, since itallows only a relatively narrow band of IR radiation to reach the sensormodule 402 from a relatively wide field of view.

[0103] Reference is now made to FIGS. 5A & 5B, which are simplifieddiagrammatic illustrations of the operation of imaging functionalityselection in accordance with another preferred embodiment of the presentinvention in two different modes of operation. In this embodiment,imaging functionality selection is effected electronically, without theneed for mechanically movable optical elements.

[0104] As seen in FIGS. 5A & 5B, this implementation of the inventioninvolves the use of an electronic imaging sensor module 502 including aCMOS detector array 504 overlaid with a repeating array of pixelwiseselective light transmissive filters 506 characterized in that the arrayincludes red (R), green (G), blue (B) and infrared (IR) transmissivefilters. A lens 508 focuses light from an imaging field onto detectorarray 504.

[0105]FIG. 5A shows the imaging functionality of the embodiment of FIGS.5A and 5B in a picture taking mode, wherein only the outputs of thepixels underlying the R, G and B filters are employed and the respectiveIR content of the outputs of each of the pixels underlying the R, G andB filters is subtracted therefrom. This color processing prevents IRradiation from distorting the color response of the sensor module 502.

[0106]FIG. 5B shows the imaging functionality of the embodiment of FIGS.5A and 5B in a data entry mode, wherein only the outputs of the pixelsunderlying the IR filters are employed.

[0107] Reference is now made to FIG. 6, which is a simplified andgeneralized illustration of a projection keyboard system and methodologywhich is preferably incorporated into the embodiments of FIGS. 1A-5B inaccordance with a preferred embodiment of the present invention. Aprojection subsystem 550, preferably including a solid state lightsource (not shown) which illuminates a spatial light modulation element(not shown), defines an image 552 of a keyboard on a projection surface554, preferably an inert surface, such as a desktop.

[0108] An illumination subsystem 556, preferably including a solid statelight source (not shown), directs light in a radially directedillumination pattern 558, which preferably extends in a plane generallyparallel to the projection surface 554. It is appreciated that theradially directed illumination pattern 558 has a very narrow spread inthe direction generally perpendicular to the projection surface 554. Itis further appreciated that the radially directed illumination pattern558 is preferably located very close to the projection surface 554.

[0109] Impingement of light in the radially directed illuminationpattern 558 on a data entry object 560, such as a user's finger, astylus or other implement, causes light to be scattered or reflectedtherefrom. It is appreciated that the light is only scattered orreflected when the data entry object 560 is in close contact with thekeyboard 552 defined on projection surface 554.

[0110] A detection subsystem 562, preferably employing a solid stateimaging sensor (not shown) incorporated in a camera in the embodimentsof FIGS. 1A-5B, receives light reflected or scattered from data entryobject 560. The received light is mapped onto the imaging sensor withrespect to a representation of the keyboard for associating the locationof the data entry object 560 sensed by detection subsystem 562 with akey location 563 on the keyboard 552. Thus, the sensed location of dataentry object 560 indicates which key of the keyboard 552 is beingengaged.

[0111] Reference is now made to FIG. 7, which is a simplified flow chartillustrating operation of a data entry object engagement locationsensing subsystem of FIG. 6 in accordance with a preferred embodiment ofthe present invention, and to FIG. 8, which is a simplified illustrationof a preferred data structure employed in the operation of the dataentry object engagement location sensing subsystem shown in FIG. 7.

[0112]FIG. 8 shows a simplified illustration of a preferred datastructure employed in the operation of the data entry object engagementlocation sensing method described hereinbelow with respect to FIG. 7. Itis appreciated that the imaging sensor of a camera, such as camera 100(FIG. 1A), is typically comprised of a set of M x N pixels, wherein aparticular group of pixels views a defined region of the engagementplane which preferably overlies the projection surface 110 (FIG. 1A).Thus, it is possible that a particular pixel group, located in the imageplane of the camera 100, may receive scattered light from a data entryobject 560 (FIG. 6) engaging the key location 563 (FIG. 6).

[0113] Thus, as the camera 100 views the projection surface 110, each ofthe M x N pixels in the image plane of the camera 100 may receive lightfrom a corresponding region in the engagement plane in respect of a dataentry object engagement therewith.

[0114] As seen in FIG. 7, pixel values, such as gray level values, areacquired for various pixel coordinates. As each pixel value is acquired,a determination is made, using the pixel coordinates, as to whether thatpixel lies within a predefined keystroke region, such as keystrokeregions 600 shown in FIG. 9. This determination is preferably made byemploying a pixel index table 601 which indicates, for each pixel,whether that pixel lies within a predetermined keystroke region, such askeystroke regions 605, 606, 607 and 608 (FIG. 9), and, if so, withinwhich keystroke region it lies.

[0115] The function of the pixel within the keystroke region in which itlies is then determined, preferably by employing table 601. Thisfunction is typically additive or subtractive, but may alternativelyhave another function. Typically, depending on the function, the pixelvalue is added to or subtracted from a pixel total maintained for eachkeystroke region in a keystroke region accumulator table 602.

[0116] Once all of the pixels in a frame have been processed asaforesaid, an updated background level is determined for the frame and akey actuation threshold is determined typically by subtracting theupdated background level from a predetermined threshold level which isestablished for each keystroke region. This is preferably carried out byemploying a keystroke region threshold table 604.

[0117] The contents of the keystroke region accumulator table 602 foreach keystroke region preferably are then compared with the current keyactuation threshold. If the contents of the accumulator table 602 exceedthe key actuation threshold for a given key actuation region in a givenframe and in the previous frame the contents of the accumulator table602 did not exceed the key actuation threshold, a key actuation outputis provided.

[0118] Similarly, if the contents of the accumulator table 602 do notexceed the key actuation threshold for a given key actuation region in agiven frame and in the previous frame the contents of the accumulatortable 602 did exceed the key actuation threshold, a key deactuationoutput is provided. In all other cases, no output need be generated.

[0119] Reference is now made to FIG. 9, which is a simplified pictorialillustration of outlines of typical keystroke regions 605, 606, 607 and608 as sensed by a two-dimensional image sensor (FIG. 6) viewing akeyboard, such as the keyboard 552, seen in FIG. 6.

[0120] Reference is now made to FIG. 10, which is a simplified pictorialillustration of outlines of typical footprints 609, 610, 611 and 612 ofa typical light pattern occasioned by data entry object engagementcorresponding to the keystroke regions 605, 606, 607 and 608 (FIG. 9).

[0121] Reference is now made to FIGS. 11A, 11B and 11C, which aresimplified illustrations of three alternative methodologies fordetermining the function of the pixel within the keystroke region inwhich it lies as shown in FIG. 8, and to FIGS. 12A, 12B and 12C, whichare simplified illustrations of traces which are useful in understandingFIGS. 11A, 11B and 11C.

[0122] Turning now to FIG. 1A, there is shown arranged along a commonarbitrary axis 618 a simplified keystroke region 620 corresponding to agiven key and containing a plurality of pixels 622. A typical simplifiedfootprint of a typical light pattern occasioned by data entry objectengagement with the given key is indicated by reference numeral 624.FIG. 10 shows outlines of typical footprints 609, 610, 611 and 612,corresponding to keystroke regions designated 605, 606, 607 and 608 inFIG. 9.

[0123] A typical background signal pattern is indicated by referencenumeral 634. Superimposition of the footprint 624 over the backgroundsignal pattern 634 is indicated at reference numeral 635. A onedimensionally selectable overlap of footprint 624 over keystroke region620 is indicated at reference numeral 636. A one dimensionallyselectable overlap of background signal pattern 634 over keystrokeregion 620 is indicated at reference numeral 637. A one dimensionallyselectable overlap of superimposition 635 over keystroke region 620 isindicated at reference numeral 638.

[0124]FIG. 12A illustrates a bias function 640 corresponding to a crosssection of the keystroke region 620 taken along axis 618, which biasfunction defines keystroke region 620 along axis 618. There is also seena signal function 644 corresponding to a cross section of the footprint624 along axis 618; a background signal function 646 corresponding to across section of the background signal pattern 634 along axis 618 and acombination signal 648 corresponding to a cross section of thesuperimposition 635.

[0125] There is also shown in FIG. 12A a one dimensional convolution 650corresponding to one dimensionally selectable overlap 636; a onedimensional convolution 652 corresponding to one dimensionallyselectable overlap 637 and a one dimensional convolution 654corresponding to one dimensionally selectable overlap 638. Convolution650 is shown with a threshold 660; convolution 652 is shown with athreshold 662 and convolution 654 is shown with a threshold 664.

[0126] Turning now to FIG. 11B, there is shown arranged along a commonarbitrary axis 670 a simplified keystroke region 680 corresponding to agiven key and containing a plurality of pixels 682. The simplifiedkeystroke region 680 is here shown surrounded by a simplified keystrokeregion border 683. A typical simplified footprint of a typical lightpattern occasioned by data entry object engagement with the given key isindicated by reference numeral 684. A typical background signal patternis indicated by reference numeral 686. Superimposition of the footprint684 over the background signal pattern 686 is indicated at referencenumeral 688. A one dimensionally selectable overlap of footprint 684over keystroke region 680 and border 683 is indicated at referencenumeral 690. A one dimensionally selectable overlap of background signalpattern 686 over keystroke region 680 and border 683 is indicated atreference numeral 692. A one dimensionally selectable overlap ofsuperimposition 688 over keystroke region 680 and border 683 isindicated at reference numeral 694.

[0127]FIG. 12B illustrates a bias function 700 corresponding to a crosssection of the keystroke region 680 and of the border 683 taken alongaxis 670, which bias function defines keystroke region 680 along axis670. It is seen that border 683 is assigned a negative value relative tothe value of the keystroke region 680. This value assignment isappreciated to enhance the value of data entry object engagements to theextent that they lie within the keystroke region 680 and to decrease thevalue of such data entry object engagements to the extent that theyextend over the border 683. There is also seen a signal function 704corresponding to a cross section of the footprint 684 along axis 670; abackground signal function 706 corresponding to a cross section of thebackground signal pattern 686 along axis 670 and a combination signal708 corresponding to a cross section of the superimposition 688.

[0128] There is also shown in FIG. 12B a one dimensional convolution720, corresponding to one dimensionally selectable overlap 690; a onedimensional convolution 722, corresponding to one dimensionallyselectable overlap 692 and a one dimensional convolution 724corresponding to one dimensionally selectable overlap 694. Convolution720 is shown with a threshold 726; convolution 722 is shown with athreshold 727 and convolution 724 is shown with a threshold 728.

[0129] Turning now to FIG. 11C, there is shown arranged along a commonarbitrary axis 730 a simplified keystroke region 740 corresponding to agiven key and containing a plurality of pixels 741. The simplifiedkeystroke region 740 is here shown surrounded by a simplified keystrokeregion inner border 742 and by a simplified keystroke region outerborder 743. A typical simplified footprint of a typical light patternoccasioned by data entry object engagement with the given key isindicated by reference numeral 744. A typical background signal patternis indicated by reference numeral 746. Superimposition of the footprint744 over the background signal pattern 746 is indicated at referencenumeral 748. A one dimensionally selectable overlap of footprint 744over keystroke region 740 and borders 742 and 743 is indicated atreference numeral 750. A one dimensionally selectable overlap ofbackground signal pattern 746 over keystroke region 740 and borders 742and 743 is indicated at reference numeral 752. A one dimensionallyselectable overlap of superimposition 748 over keystroke region 740 andborders 742 and 743 is indicated at reference numeral 754.

[0130]FIG. 12C illustrates a bias function 760 corresponding to a crosssection of the keystroke region 740 and of the borders 742 and 743 takenalong axis 730, which bias function defines keystroke region 740 alongaxis 730. It is seen that border 742 is assigned a zero value and border743 is assigned a negative value relative to the value of the keystrokeregion 740. It is appreciated that these value assignments enhance thevalue of data entry object engagements that to the extent that they liewithin the keystroke region 740 and to decrease the value of such dataentry object engagements to the extent that they extend across theborder 742 and at least into border 743. There is also seen a signalfunction 764 corresponding to a cross section of the footprint 744 alongaxis 730; a background signal function 766 corresponding to a crosssection of the background signal pattern 746 along axis 730 and acombination signal 768 corresponding to a cross section of thesuperimposition 748.

[0131] There is also shown in FIG. 12C a one dimensional convolution780, corresponding to one dimensionally selectable overlap 750; a onedimensional convolution 782, corresponding to one dimensionallyselectable overlap 752 and a one dimensional convolution 784corresponding to one dimensionally selectable overlap 754. Convolution780 is shown with a threshold 786; convolution 782 is shown with athreshold 787 and convolution 784 is shown with a threshold 788.

[0132] It may be appreciated from a consideration of convolutions 638,694 and 754 that the dual border arrangement of FIGS. 11C and 12Cprovides the best detection of data entry object keystroke engagement,while minimizing background effects.

[0133] Reference is now made to FIG. 13, which is a simplified flowchart illustrating operation of a data entry object engagement locationsensing subsystem employed in a tracking system and methodologyconstructed and operative in accordance with a preferred embodiment ofthe present invention.

[0134] As seen in FIG. 13, pixel values, such as gray level values, areacquired for various pixel coordinates. As each pixel value is acquired,a determination may be made, using the pixel coordinates, as to whetherthat pixel lies within a predefined active region. Typically, if thepixel does not lie within a predetermined active region, its value isignored.

[0135] The pixel value for each pixel is preferably thresholded andtypically all pixel values falling below a given threshold are ignored.The remaining pixel values may be weighted by a selected weightingparameter. In order to determine the “center of gravity” of thethresholded and weighted pixel values, the thresholded and weightedpixel values are multiplied respectively by X and Y values representingthe coordinate position of each pixel and the results are summed alongmutually perpendicular axes X and Y and stored in X and Y accumulators.The total of the thresholded and weighted pixel values for all relevantpixels are also summed and stored in a data accumulator, for the entireactive region.

[0136] Once all of the pixels in a frame have been processed asaforesaid, the summed thresholded and weighted pixel values along the Xand Y axes respectively are divided by the total of the thresholded andweighted pixel values for the entire active region to determine the Xand Y coordinates of the “center of gravity”, which represents a desiredengagement location.

[0137] Reference is now made to FIG. 14, which is a simplifiedillustration showing synchronized illumination power variationfunctionality useful in accordance with a preferred embodiment of thepresent invention. The functionality illustrated in FIG. 14 is directedto modulating the amount of illumination provided for data entry objectengagement detection. This modulation is desirable because the intensityof light impinging on a data entry object and thus scattered thereby,decreases with the distance between an illuminator 830 and a data entryobject. Thus, it may be appreciated that the amount of light impingingon a data entry object 832 is substantially greater than the amount oflight impinging on a data entry object 834. Furthermore, the amount ofscattered light impinging on a detector 836 decreases with the distancebetween the data entry object and the detector. These two distancedependencies make detection of data entry object engagement difficult.

[0138] In order to overcome this difficulty, there is provided inaccordance with a preferred embodiment of the present invention variableintensity drive electronics 840 which is coupled to both illuminator 830and detector 836 and which causes the intensity of light produced by theilluminator 830 to vary, typically in a ramp fashion, in synchronizationto the imaging field location of detector 836.

[0139] Thus, it may be seen that when a near portion (A) of theengagement surface 554 (FIG. 6) is being imaged, typically at the topportion A of detector 836, the intensity is at a minimum. When anintermediate portion (B) of the engagement surface 554 is being imaged,typically at the middle of detector 836, the intensity is at anintermediate level and when a far portion (C) of the engagement surface554 is being imaged, typically at the bottom portion (C) of the detector836, the intensity is at a maximum.

[0140] Variable intensity drive electronics 840 operates preferably byproviding a synchronization output 842 to detector 836 and acorresponding synchronization output 844 to illuminator 830, causing theintensity level to increase in time in synchronization with the locationof a scanned image region in detector 836.

[0141] Reference is now made to FIG. 15, which is a simplified partiallypictorial, partially diagrammatic illustration of data entry apparatusembodied in a vehicle access control system. As seen in FIG. 15, avehicle 1500, typically comprising a frame, a drive train and a chassisincluding at least one lockable entry portal or door 1502 and portalunlocking functionality, includes an access control assembly designatedgenerally by reference numeral 1504.

[0142] The access control assembly 1504 preferably includes at least oneat least partially light transmissive user finger engagement surfaceaccessible from outside the chassis. In the illustrated embodiment,plural user finger engagement surfaces 1506 are defined on a car doorwindow 1508. An illuminator 1510, preferably located interiorly of theexterior of the vehicle, such as an IR line generator, is operative toilluminate a finger engagement plane 1512 adjacent the user fingerengagement surfaces 1506. A preferred IR line generator is described inapplicant's Published PCT Application WO02/054169 A2, the disclosure ofwhich is hereby incorporated by reference.

[0143] A two-dimensional imaging sensor 1514, located interiorly of theexterior of the vehicle, views the finger engagement surfaces 1506, froma location inwardly of the car door window 1508, for sensing lightoriginating from the illuminator 1510, which is scattered by engagementof a user's finger with the finger engagement surfaces 1506. A preferredtwo-dimensional imaging sensor 1514 is described in applicant'sPublished PCT Application WO02/054169 A2, the disclosure of which ishereby incorporated by reference.

[0144] A data entry processor 1516 receives an output from thetwo-dimensional imaging sensor 1514 and provides a data entry input to aconventional door unlocking functionality (not shown).

[0145] Reference is now made to FIG. 16, which is a simplifieddiagrammatic illustration of a variation in the data entry apparatus ofFIG. 15. As seen in FIG. 16, a vehicle 1600, typically comprising aframe, a drive train and a chassis including at least one lockable entryportal or door 1602 and portal unlocking functionality, includes anaccess control assembly designated generally by reference numeral 1604.

[0146] The access control assembly 1604 preferably includes at least oneat least partially light transmissive user finger engagement surfaceaccessible from outside the chassis. In the illustrated embodiment,plural user finger engagement surfaces 1606 are defined on a car doorwindow 1608. In this embodiment, the plural user finger engagementsurfaces 1606 may be arranged in a two-dimensional array. An illuminator1610, preferably located interiorly of the exterior of the vehicle, suchas an IR line generator, is operative to illuminate a finger engagementplane 1612 adjacent the user finger engagement surfaces 1606. Apreferred IR line generator is described in applicant's Published PCTApplication WO02/054169 A2, the disclosure of which is herebyincorporated by reference.

[0147] In the embodiment of FIG. 16, the finger engagement planes 1612are parallel to the finger engagement surfaces 1606, by virtue ofprovision of a prism 1613 disposed typically on the outside of thevehicle window 1608 along the light path from illuminator 1610.

[0148] A two-dimensional imaging sensor 1614, located interiorly of theexterior of the vehicle, views the finger engagement surfaces 1606, froma location inwardly of the car door window 1608, for sensing lightoriginating from the illuminator 1610, which is scattered by engagementof a user's finger with the finger engagement surfaces 1606. A preferredtwo-dimensional imaging sensor 1614 is described in applicant'sPublished PCT Application WO02/054169 A2, the disclosure of which ishereby incorporated by reference.

[0149] A data entry processor 1616 receives an output from thetwo-dimensional imaging sensor 1614 and provides a data entry input to aconventional door unlocking functionality (not shown).

[0150] It is appreciated that portions of the apparatus of FIGS. 15 and16 may find application in other types of access control environmentsand in various data entry environments.

[0151] Reference is now made to FIG. 17, which is a simplified pictorialillustration of data entry apparatus constructed and operative inaccordance with another preferred embodiment of the present invention.As seen in FIG. 17, a light emitting data entry engagement element,preferably in the form of a stylus 1700, emits light when and only whenit is in at least predetermined propinquity to an engagement surface1702.

[0152] A two-dimensional imaging sensor 1704 views the engagementsurface 1702 for sensing light emitted by the stylus 1700. A data entryprocessor 1706 receives an output from the two-dimensional imagingsensor 1704 and provides a data entry input.

[0153] As seen in FIG. 17, the stylus 1700 preferably comprises an IRemitting LED 1708 located in an IR transmissive tip 1710 thereof. Aproximity switch 1712, such as a contact switch, provides an outputindication of the proximity of tip 1710 to engagement surface 1702.Circuitry 1714 receives an input from proximity switch 1712 and operatesthe LED 1708 accordingly, so as to provide an IR emission whose locationindicates the location of stylus impingement with engagement surface1702.

[0154] Reference is now made to FIGS. 18A and 18B, which arerespectively a simplified pictorial and a simplified sectionalillustration of data entry apparatus constructed and operative inaccordance with another preferred embodiment of the present invention.As seen in FIGS. 18A & 18B, the data entry apparatus comprises aprojector 1800 operative to illuminate at least one engagement surface1802 and to define thereon at least one touchpad region 1804 and atleast one keyboard region 1806, the touchpad region 1804 being definedby a zero'th order diffracted image provided by the projector 1800.

[0155] Turning particularly to FIG. 18B, it is seen that the projector1800 comprises a light source, such as a diode laser 1810, which outputsvia a collimating lens 1812 onto a first diffractive optical element1814. Diffractive optical element 1814 diffracts the light impingingthereon into a primary keyboard image beam 1816 which impinges on atlast one keyboard region 1806 and into a secondary ghost image beam1818, which is preferably blocked by a barrier 1819. Residual lightpassing through diffractive optical element 1814 defines a zero'th orderbeam 1820, which impinges onto a second diffractive optical element1822.

[0156] Diffractive optical element 1822 diffracts the light impingingthereon from the zero'th order beam 1820 into a touchpad defining beam1824 which impinges on at least one touchpad region 1804.

[0157] Returning to FIG. 18A, it is seen that there is also provided anilluminator 1830, operative to illuminate the finger engagement surface.A two-dimensional imaging sensor 1832 views the engagement surface forsensing light from the illuminator 1830 scattered by engagement of auser's finger with the finger engagement surface. A data entry processor1834 receives an output from the two-dimensional imaging sensor 1832 andprovides a data entry input.

[0158] Reference is now made to FIGS. 19A and 19B, which arerespectively a simplified pictorial and a simplified sectionalillustration of data entry apparatus constructed and operative inaccordance with yet another preferred embodiment of the presentinvention. As seen in FIGS. 19A & 19B, the data entry apparatuscomprises a projector 1900 operative to illuminate at least one fingerengagement surface 1902 and to define thereon typically at least onekeyboard region 1904. An illuminator 1906 is operative to illuminate thefinger engagement surface 1902.

[0159] An imaging sensor module 1908, including a two-dimensionalimaging sensor 1910, views the engagement surface for sensing light fromthe illuminator 1906 scattered by engagement of a user's finger with thefinger engagement surface 1902. A particular feature of the imagingsensor module 1908 is that it includes an optical wedge element 1912which provides a tilt, enabling a generally vertically disposed imagingsensor 1910 to efficiently sense light from a finger engagement surface1902 extending generally perpendicularly thereto.

[0160] A data entry processor 1914 receives an output from thetwo-dimensional imaging sensor 1910 and provides a data entry input.

[0161] Reference is now made to FIGS. 20A and 20B, which arerespectively a simplified pictorial and a simplified sectionalillustration of data entry apparatus constructed and operative inaccordance with still another preferred embodiment of the presentinvention. As seen in FIGS. 20A & 20B, the data entry apparatuscomprises a projector 2000 operative to illuminate at least one fingerengagement surface 2002 and to define thereon typically at least onekeyboard region 2004. An illuminator 2006 is operative to illuminate thefinger engagement surface 2002.

[0162] An imaging sensor module 2008, including a two-dimensionalimaging sensor 2010, views the engagement surface for sensing light fromthe illuminator 2006 scattered by engagement of a user's finger with thefinger engagement surface 2002. A particular feature of the imagingsensor module 2008 is that it includes a pair of oppositely orientedoptical wedge elements 2012 and 2014, each of which provides a tilt to aportion of the light incoming to sensor 2010, enabling efficient use ofarea of generally rectangular imaging sensor, as seen at referencenumber 2016.

[0163] Reference is now made to FIG. 21, which is a simplified pictorialillustration of data entry apparatus constructed and operative inaccordance with yet another preferred embodiment of the presentinvention. As seen in FIG. 21, the data entry apparatus comprises aprojector 2100 operative to illuminate at least one finger engagementsurface 2102 and to define thereon typically at least one keyboardregion 2104. An illuminator 2106 is operative to illuminate the fingerengagement surface 2102.

[0164] An imaging sensor module 2108, including a two-dimensionalimaging sensor 2110, views the engagement surface for sensing light fromthe illuminator 2106 scattered by engagement of a user's finger with thefinger engagement surface 2102.

[0165] A particular feature of the illuminator 2106 is that defines anauxiliary light path 2112 which impinges on a proper positioningconfirmation location 2114 on the finger engagement surface 2102adjacent and preferably nearer to the illuminator than the at least onekeyboard region 2104. The imaging sensor module 2108 senses the presenceof illumination at location 2114 and in the absence of sensedillumination at that location, typically indicating improper placementof the data entry apparatus, typically unpowers the data entryapparatus.

[0166] A data entry processor 2116 receives an output from thetwo-dimensional imaging sensor 2110 and provides a data entry input.

[0167] Reference is now made to FIG. 22, which is a simplified pictorialillustration of data entry apparatus constructed and operative inaccordance with another preferred embodiment of the present invention inthe form of a wristwatch. As seen in FIG. 22, a wristwatch 2200, whichmay be of conventional construction other than as specifically describedhereinbelow, is equipped with an electronic imaging sensor module 2202,which may include a CMOS array, for example, CMOS array OVT6130,available from Omnivision Technologies, Inc., 1341 Orleans Drive,Sunnyvale, Calif., USA, providing an output representing an imagedfield.

[0168] A user-operated imaging functionality selection switch 2204 ispreferably provided to enable a user to select operation in a data entryfunctionality employing the electronic imaging sensor module 2202 fordata entry responsive to user hand activity.

[0169] The data entry functionality preferably employs a data entrytemplate projector 2206, which projects a data entry template, such asthat designated by reference numeral 2208, onto a surface, such as atable surface 2210. An illuminator 2212 is operative to illuminate aregion adjacent the data entry template. The imaging sensor module 2202views the data entry template region for sensing light from theilluminator 2212 scattered by engagement of a user's finger with thedata entry template.

[0170] The data entry functionality also preferably incorporatesprocessing functionality, an example of which is described inapplicant's Published PCT Application WO 02/054169 A2, the disclosure ofwhich is hereby incorporated by reference. The data entry functionalityproduces a stream of data 2214.

[0171] Reference is now made to FIG. 23, which is a simplified pictorialillustration of gaming apparatus constructed and operative in accordancewith a preferred embodiment of the present invention. As seen in FIG.23, gaming apparatus, one example of which is a board game, such aschess, is provided with an electronic imaging sensor module 2302, whichmay include a CMOS array, for example, CMOS array OVT6130, availablefrom Omnivision Technologies, Inc., 1341 Orleans Drive, Sunnyvale,Calif., USA, providing an output representing an imaged field.

[0172] A user-operated imaging functionality selection switch 2304 ispreferably provided to enable a user to select operation of one ofplural gaming functionalities, such as different board games, employingthe electronic imaging sensor module 2302 for gaming piece locating.

[0173] The gaming functionality preferably employs a gaming templateprojector 2306, which projects a game template, such as a chess board,designated by reference numeral 2308, onto a surface, such as a tablesurface 2310. An illuminator 2312 is operative to illuminate a regionadjacent the game template 2308. The imaging sensor module 2302 viewsthe game template region for sensing light from the illuminator 2312reflected by optically encoded reflectors 2314 mounted on individualgaming pieces 2316. The reflectors 2314 preferably individually identifyeach gaming piece and indicate its location on the game template 2308.

[0174] The gaming functionality also preferably incorporates processingfunctionality, an example of which is described in applicant's PublishedPCT Application WO 02/054169 A2, the disclosure of which is herebyincorporated by reference. The gaming functionality produces a stream ofdata 2316, which may be used for game management and scoring.

[0175] Reference is now made to FIG. 24, which is a simplified pictorialillustration of gaming apparatus constructed and operative in accordancewith another preferred embodiment of the present invention. FIG. 24shows gaming pieces, such as dice 2402, which bear variously differentlycoded reflectors 2404 on various surfaces thereof, thereby to enable notonly the identity and location but also the spatial orientation of thegaming pieces to be sensed.

[0176] It will be appreciated by persons skilled in the art that thepresent invention is not limited to what has been particularly shown anddescribed hereinabove. Rather the scope of the present inventionincludes both combinations and subcombinations of the various featuresdescribed hereinabove as well as modifications and variations thereof aswould occur to a person of skill in the art upon reading the foregoingspecification and which are not in the prior art.

1. An electronic camera comprising: an electronic imaging sensorproviding an output representing an imaged field; a first imagingfunctionality employing said electronic imaging sensor for taking apicture of a scene in said imaged field; a second imaging functionalityemploying said electronic imaging sensor for data entry responsive touser hand activity; and a user-operated imaging functionality selectionswitch operative to enable a user to select operation in one of saidfirst and second imaging functionalities.
 2. An electronic cameraaccording to claim 1 and wherein said electronic imaging sensorcomprises a CMOS sensor.
 3. An electronic camera according to claim 1and wherein said first imaging functionality incorporates a first lensarrangement upstream of said electronic imaging sensor.
 4. An electroniccamera according to claim 1 and wherein said second imagingfunctionality incorporates a second lens arrangement upstream of saidelectronic imaging sensor.
 5. An electronic camera according to claim 1and wherein said second imaging functionality incorporates an infraredlight generator for defining a light beam, impingement of whichresulting from said user hand activity produces a light patternrepresentative of said data.
 6. An electronic camera according to claim5 and wherein said second imaging functionality also incorporates atleast one infrared passing light filter associated with said electronicimaging sensor.
 7. An electronic camera according to claim 1 and whereinsaid second imaging functionality incorporates a processor receiving anoutput from said electronic imaging sensor for providing a data entryoutput.
 8. An electronic camera according to claim 7 and wherein saidprocessor is incorporated together with said electronic imaging sensoron a single chip.
 9. An electronic camera according to claim 1 andwherein said first functionality is a color photography functionality.10. An electronic camera according to claim 1 and wherein said secondfunctionality is an infrared beam impingement sensing data entryfunctionality.
 11. An electronic camera according to claim 1 and whereinsaid user-operated imaging functionality selection switch is amechanically responsive switch which selectably mechanically associatessaid first and second lens arrangements upstream of said electronicimaging sensor.
 12. An electronic camera according to claim 1 andwherein said user-operated imaging functionality selection switch is amechanically responsive switch which selectably mechanically associatesselected filters with said electronic imaging sensor.
 13. An electroniccamera according to claim 1 and wherein said user-operated imagingfunctionality selection switch is an electronically responsive switchwhich selectably electronically determines responsiveness to selectedoutputs from predetermined portions of said electronic imaging sensor.14. An electronic camera according to claim 1 and comprising an array ofred, green, blue and infrared passing pixelwise elements arranged oversaid electronic imaging sensor.
 15. An electronic camera according toclaim 1 and also comprising a data entry template projector employed bysaid second imaging functionality.
 16. An electronic camera according toclaim 1 and also comprising an illuminator employed by said secondimaging functionality.
 17. An electronic camera according to claim 1 andalso comprising a synchronized illumination power variationfunctionality.
 18. An electronic camera comprising: an electronicimaging sensor providing an output representing an imaged field; and anarray of red, green, blue and infrared passing pixelwise elementsarranged over said electronic imaging sensor.
 19. An electronic cameracomprising: an electronic imaging sensor providing an outputrepresenting an imaged field; a first imaging functionality employingsaid electronic imaging sensor in a visible radiation imaging mode fortaking a picture of a scene in said imaged field; and a second imagingfunctionality employing said electronic imaging sensor in an infraredradiation sensing mode for data entry responsive to user hand activity.20. An electronic camera according to claim 19 and comprising an arrayof red, green, blue and infrared passing pixelwise elements arrangedover said electronic imaging sensor.
 21. An electronic camera accordingto claim 20 and wherein in said first imaging functionality output ofthe pixels underlying the red, green and blue passing elements areemployed.
 22. An electronic camera according to claim 21 and whereinoutput of the pixels underlying said infrared passing elements isemployed to prevent distortion of color response of said imaging sensor.23. An electronic camera according to claim 20 and wherein in saidsecond imaging functionality output of the pixels underlying theinfrared passing elements are employed.
 24. An electronic cameraaccording to claim 20 and also comprising a processor.
 25. An electroniccamera according to claim 1 and wherein said electronic camera formspart of a telephone.
 26. An electronic camera according to claim 1 andwherein said electronic camera forms part of a personal digitalassistant.
 27. An electronic camera according to claim 1 and whereinsaid electronic camera forms part of a wristwatch. 28-38. (Cancelled)39. A portable telephone comprising: telephone functionality; anelectronic imaging sensor providing an output representing an imagedfield; a first imaging functionality employing said electronic imagingsensor for taking a picture of a scene in said imaged field; a secondimaging functionality employing said electronic imaging sensor for dataentry responsive to user hand activity; and a user-operated imagingfunctionality selection switch operative to enable a user to selectoperation in one of said first and second imaging functionalities.40-57. (Cancelled)
 58. A portable telephone comprising: telephonefunctionality; an electronic imaging sensor providing an outputrepresenting an imaged field; a first imaging functionality employingsaid electronic imaging sensor in a visible radiation imaging mode fortaking a picture of a scene in said imaged field; a second imagingfunctionality employing said electronic imaging sensor in an infraredradiation sensing mode for data entry responsive to user hand activity;and an array of red, green, blue and infrared passing pixelwise elementsarranged over said electronic imaging sensor. 59-73. (Cancelled)
 74. Adigital personal assistant comprising: at least one personal digitalassistant functionality; an electronic imaging sensor providing anoutput representing an imaged field; a first imaging functionalityemploying said electronic imaging sensor for taking a picture of a scenein said imaged field; a second imaging functionality employing saidelectronic imaging sensor for data entry responsive to user handactivity; and a user-operated imaging functionality selection switchoperative to enable a user to select operation in one of said first andsecond imaging functionalities. 75-92. (Cancelled)
 93. A digitalpersonal assistant comprising: at least one personal digital assistantfunctionality; an electronic imaging sensor providing an outputrepresenting an imaged field; a first imaging functionality employingsaid electronic imaging sensor in a visible radiation imaging mode fortaking a picture of a scene in said imaged field; a second imagingfunctionality employing said electronic imaging sensor in an infraredradiation sensing mode for data entry responsive to user hand activity;and an array of red, green, blue and infrared passing pixelwise elementsarranged over said electronic imaging sensor. 94-148. (Cancelled) 149.An electronic camera according to claim 20 and wherein said electroniccamera forms part of a telephone.
 150. An electronic camera according toclaim 20 and wherein said electronic camera forms part of a personaldigital assistant.
 151. An electronic camera according to claim 20 andwherein said electronic camera forms part of a wristwatch.